Layered Learning for a Soccer Legged Robot Helped with a 3D Simulator

In the robotic soccer domain, many factors, such as the speed of individual robots, the effectiveness of kicks, and the choice of the appropriate attacking strategy, determine the success of a team. Consequently, soccer robots, and in particular legged ones, require fine tuning of the parameters not only for the vision processes, but also in the implementation of behaviors and basic control actions and in the strategic decisional processes. In recent years, machine learning techniques have been used to find optimal parameter sets for different behaviors. In particular, layered learning has been proposed to improve learning rate in robot learning tasks. In this paper we consider a layered learning approach for learning optimal parameters of basic control routines, behaviours and strategy selection. We compare three different methods in the different layers: genetic algorithm, Nelder-Mead, and policy gradient. Moreover, we study how to use a 3D simulator for speeding up robot learning. The results of our experimental work on AIBO robots are useful not only to state differences and similarities between different robot learning approaches used within the layered learning framework, but also to evaluate a more effective learning methodology that makes use of a simulator.